HTTP, Web, and JSON

CIS 193 – Go Programming

Prakhar Bhandari, Adel Qalieh

CIS 193

Course Logistics

HTTP

HTTP (Hyper Text Transfer Protocol) is a client-server protocol. Remember that a server is an application that listens for incoming requests from clients, and returns and appropriate response.

When you access a page on the web, you make an HTTP request to the webserver hosting the page, and you get the HTML from the server as a response.

HTTP has verbs such as GET, POST, DELETE, etc.

HTTP Client

Making HTTP requests as a client in Go is simple. The http package is full of methods for making HTTP requests. 

resp, err := http.Get("http://example.com/")
...
resp, err := http.Post("http://example.com/upload", "image/jpeg", &buf)
...
resp, err := http.PostForm("http://example.com/form",
  url.Values{"key": {"Value"}, "id": {"123"}})

Variable resp is of type *Response, which has a Status, Header, Body, etc. 

resp.Status        // "200 OK"
resp.StatusCode    // 200
resp.Proto         // "HTTP/1.0"

Reading HTTP Body

The response body (resp.Body) is an io.ReadCloser. To read the body, you must treat it like any other type with a Reader, like files. 

type ReadCloser interface {
  Reader
  Closer
}

After you are done reading the Body, you must Close() it. Why?

HTTP Client Demo

HTTP Server

Making a barebones HTTP server in Go is just as easy. The http package also comes with a fully-featured HTTP server. 

http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
})

log.Fatal(http.ListenAndServe(":8080", nil))

ResponseWriter has a Write([]byte) method. What interface does it fulfill automatically?

HTTP Server Demo

Web

The web is made up of HTML documents that are served over HTTP. We can use HTML responses and do HTML templating with the built-in template package. 

var homeTemplate = template.Must(template.ParseFiles("home.html"))

func upload(w http.ResponseWriter, r *http.Request) {
  homeTemplate.Execute(w, nil)
}

Web Page Demo

JSON

Example: 

{
  "id": 1,
  "name": "A green door",
  "price": 12.50,
  "tags": ["home", "green"]
}

The equivalent native type in Go would be: 

type Door struct {
  ID    int
  Name  string
  Price float64
  Tags  []string
}

Motivation

The goal is to convert this 

{
  "name":"Gopher",
  "birthdate": "2017/02/28",
  "shirt-size": "XS"
}

into this 

type Person struct {
  Name string
  Born time.Time
  Size string
}

Simple JSON Parsing

{
  "id": 1,
  "name": "A green door",
  "price": 12.50,
  "tags": ["home", "green"]
}

Step 1: Add struct tags 

type Door struct {
    ID    int      `json:"id"`
    Name  string   `json:"name"`
    Price float64  `json:"price"`
    Tags  []string `json:"tags"`
}

This sets the key used for encoding and decoding JSON. If the tag is omitted, the default is the struct field name. (ex: "Price")

Encoding and Decoding

Use the Marshal and Unmarshal methods from the json package to convert back and forth between JSON as an array of bytes and its struct representation.

Encoding aka Marshaling 

ourDoor := &Door{
    ID:    1,
    Name:  "A gray door",
    Price: 24.99,
    Tags:  []string{"old", "engineering"},
}
jsonDoor, err := json.Marshal(ourDoor)

Decoding aka Unmarshaling 

incomingDoor := []byte(`{"id": 2, "name": "red door", "price": 99.50, "tags": []}`)
newDoor := Door{}
err := json.Unmarshal(incomingDoor, &newDoor)

JSON Parsing Demo

What if I don't know the types?

Ideally, JSON should be structured enough that type checking will help you. However, this is not always this case. 

{
  "doors": [
    {
      "id": 1,
      "name": "A green door"
    },
    {
      "id": "2",
      "name": ["white and gold door", "blue and black door"]
    }
  ]
}

Solution: use interface{}!

Whenever your data type is unknown, use an interface and runtime type conversions. 

var data map[string]interface{}
err := json.Unmarshal(incomingDoor, &data)

Remember: interface{} requires type casting. See type switching for a more sophisticated case of type conversions. 

id := data["id"]  // interface{} (wrong!)
id++  // invalid operation: id++ (non-numeric type interface {})

id := data["id"].(int)  // int (correct!)
id++  // => 3

Custom Type Conversions

json.Marshal only knows how to convert some basic native types. What about our time.Time type in the original example? 

type Person struct {
  Name string
  Born time.Time   //  "2017/02/28"
  Size string
}

Solution: Use a map[string]string, and convert types.

See the json documentation for more details.

Dates in Go

Time formatting is based on a "magic" date: 

Mon Jan 2 15:04:05 -0700 MST 2006
0   1   2  3  4  5   7          6

Date formatting usage

Simply order the "magic" date into the format that you want! 

now := time.Now()

fmt.Println(now)                         => 2017-02-28 10:44:46.584220595 -0500 EST
fmt.Println(now.Format("1/2/06"))        => 2/28/17
fmt.Println(now.Format("2006-01-02"))    => 2007-02-28

You can have arbitrary text in your date format strings as long as they don't conflict with the magic date keywords. 

fmt.Println(now.Format("Today is Monday, January 2nd!"))
fmt.Println(now.Format("Alert: it is now 3pm and 4 minutes past the hour"))

Parsing dates

Parsing dates and times uses the same formatting style. 

t, err := time.Parse("2 Jan, 3:04", "28 Feb, 10:43")

t.Year()  //  => 0

Any non-specified fields default to the zero (0) value!

Final JSON Parsing

Putting it all together, we have JSON unmarshaling, together with struct initialization, and date parsing. 

fields := map[string]string{}
err := json.Unmarshal(incomingJSON, &fields)
p := Person{}
p.Name = fields["name"]
t, err := time.Parse("2006/01/02", fields["born"])
...
p.Born = t
p.Size = fields["size"]

Using JSON with HTTP

HTTP response bodies are not string or []byte, they are io.ReadCloser.
The json package has a json.NewDecoder method that takes in an io.Reader.

The Decoder has a Decode method that works very much like Unmarshal. 

dec := json.NewDecoder(req.Body)
if err := dec.Decode(&person); err != nil {
  return fmt.Errorf("decode person: %v", err)
}

Homework 6

Thank you

Prakhar Bhandari, Adel Qalieh

CIS 193

https://cis193.com/

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)